-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG 1858400: [Performance] Lease refresh period for machine-api-controllers is too high, causes heavy writes to etcd at idle #100
BUG 1858400: [Performance] Lease refresh period for machine-api-controllers is too high, causes heavy writes to etcd at idle #100
Conversation
@elmiko: This pull request references Bugzilla bug 1858400, which is invalid:
Comment In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
@elmiko: This pull request references Bugzilla bug 1858400, which is invalid:
Comment In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/bugzilla refresh |
@elmiko: This pull request references Bugzilla bug 1858400, which is valid. The bug has been updated to refer to the pull request using the external bug tracker. 3 validation(s) were run on this bug
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
@dhellmann Do you have a moment to review? We'd need to backport this implementation into 4.5, but before that, this BZ should be fixed. |
i was going to add the backports as my next step, i had been using this bug to report under https://bugzilla.redhat.com/show_bug.cgi?id=1861896 basically i have been backporting the leader elect mechanisms and bringing all these timing changes at once. |
/retest |
The change looks fine. It sounds like there's a known issue with the CI job that isn't related to this PR. /approve |
/retest |
/lgtm |
dfb1056
to
393dbb5
Compare
updating with 90s retry time |
@elmiko: This pull request references Bugzilla bug 1858400, which is valid. 3 validation(s) were run on this bug
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Prevent machine controllers from writing in etcd at idle too often by setting 120s lease, 20s retry and 110s deadline on all renewals. Higher values cause tests to flake.
393dbb5
to
097869a
Compare
after some discussion with the team we have agreed that 20s retry period is acceptable for now. i have fixed the pr |
@elmiko: This pull request references Bugzilla bug 1858400, which is valid. 3 validation(s) were run on this bug
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/lgtm |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Danil-Grigorev, dhellmann, elmiko The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
@elmiko: Some pull requests linked via external trackers have merged: The following pull requests linked via external trackers have not merged: These pull request must merge or be unlinked from the Bugzilla bug in order for it to move to the next state. Bugzilla bug 1858400 has not been moved to the MODIFIED state. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
🏃 Add BMO pull policy
Prevent machine controllers from writing in etcd at idle too often by
setting 120s lease, 20s retry and 110s deadline on all renewals. Higher
values cause tests to flake.
Inspired by openshift/cloud-credential-operator#231
Depends on openshift/machine-api-operator#675